Hidden Representation after Reinforcement Learning of Hand Reaching Movement with Variable Link Length

Iriki et al. reported interesting results regarding the visual receptive field of two kinds of neurons in the parietal cortex of a monkey. A monkey did a task to reach its hand or tool to a target. The receptive field of one kind of neuron was enlarged when the monkey used the tool grasped by its hand. The receptive field of the other type of neurons moved together with its hand even though the hand was hidden under an opaque plate. They discussed those results in relation to high-order cognitive functions such as body image and symbolization[1]-[3]. In this paper, a hypothesis is posited that these neurons contribute to generate the critic output (state evaluation in a given task) and are obtained through reinforcement learning. Thereby, tool use is considered to be the change of link length for simplicity; a layered neural network learns hand reaching by a manipulator based on reinforcement learning. Inputs of the network are visual sensory signals and the state of the manipulator. Outputs are the critic and joint torques as the actor. After learning, the manipulator came to move its hand toward the target on the visual sensor when the target was located within the hand’s reach. Both types of neurons observed in experiments of Iriki et al. were found in the hidden layer of the neural network.